Rails Ruby on Rails

Comments and changes to this ticket

• You flagged this item as spam.
  

  Samuel Kadolph July 27th, 2010 @ 07:43 PM

  I can confirm that rails 3 rc is no longer autoloading lib folders for engines.
  Will update when I track down the change that broken this. (Or was the change intentional?)

• You flagged this item as spam.
  

  Samuel Kadolph July 27th, 2010 @ 07:54 PM

  The problem is ultimately from the configuration of Rails::Paths::Root in railties-3.0.0.rc/lib/rails/engine/configuration.rb.

  In the RC, a new option was added called :autoload. Presumably to support the new autoload features in ActiveSupport.
  The problem is that the :set_autoload_paths initializer for engines only uses autoload_paths, eager_load_paths and autoload_once_paths and not load_paths like it used to.

  I see two solutions:

  1. Add :autoload => true to paths.lib "lib" in engine/configuration.rb; or
  2. Expand Rails::Engine._all_autoload_paths to include config.load_paths
• 

  Neeraj Singh July 27th, 2010 @ 08:02 PM

  • Assigned user set to “Xavier Noria”
  • Importance changed from “” to “Low”

  If I remember correctly then Xavier made the changes which removes lib from autoloading. Not sure though.

• 

  Xavier Noria July 27th, 2010 @ 08:53 PM

  • Assigned user changed from “Xavier Noria” to “José Valim”

  Not really. Perhaps it rang a bell that I renamed load -> autoload in dependencies.rb and application.rb.

  Not autoloading lib was introduced later in http://github.com/rails/rails/commit/9b19a6f16cebf4257d2f0b839f6cc8....

• 

  José Valim July 27th, 2010 @ 09:11 PM

  • State changed from “new” to “invalid”

  Intentional.

• You flagged this item as spam.
  

  Jeff Dean July 30th, 2010 @ 05:39 AM

  Jose - could you please expand on why this change was intentional?
  

  I haven't come across an app that didn't assume files in /lib were loaded. The error messages this change causes are somewhat difficult to debug, since there's no deprecation warning that users can look at to get a clue. To me this seems like an odd change.

• 

  José Valim July 30th, 2010 @ 07:29 AM

  We've done an extensive work in making engines behaving closer to an application (and vice-versa) in Rails 3 and will continue in Rails versions. However, engines cannot autoload stuff in lib since they are usually required by the gem so we did this change to make them behave the same.

  Besides, this makes a Rails application behave more closely to any Ruby project, leaving the autoload to directories specific to Rails (but not lib).

  Finally, autoloading in lib was always causing some pain, since some people had files like "liquid_extensions.rb" but there was no "LiquidExtensions" module defined it.

  I guess these are the main reasons. :)

• You flagged this item as spam.
  

  Mateo Murphy August 1st, 2010 @ 04:48 PM

  I'm not sure about this change; I've traditionally kept things like form builders in the lib folder, expecting them to be autoloaded when I first referenced them, and I assumed this was the Rails Way of doing it. Now with this change, it makes me wonder then what the best practice is for code you want autoloaded that doesn't fit in the rails app folders?

• You flagged this item as spam.
  

  schof August 1st, 2010 @ 05:05 PM

  Another unexpected change from beta4 to RC is that an engine's app/models are no longer cached in development mode.

• 

  Xavier Noria August 1st, 2010 @ 06:28 PM

  @Mateo, re best practice: no matter whether you put stuff in lib or on some custom folder like app/observers, etc., it is totally fine that you want that stuff to be there and autoloadable, and Rails provides config.autoload_paths for that.

  This change does not mean autoloading stuff in lib is considered to be a bad practice per se. It rather has been thought to be a better default, with the rationale José explained above.

  So, organize the code in a way that makes sense to you, and use config.autoload_paths to get autoloading in non-standard places.

• You flagged this item as spam.
  

  Mateo Murphy August 1st, 2010 @ 06:39 PM

  @Xavier may I suggest then, changing the generated config/application.rb,

      # Custom directories with classes and modules you want to be autoloadable.
      # config.autoload_paths += %W(#{config.root}/extras)
  

  to something like

      # Custom directories with classes and modules you want to be autoloadable.
      # By default, the lib folder is not autoloadable.
      # config.autoload_paths += %W(#{config.root}/lib)
  

  Which would make it easier for people coming from previous version of rails to regain that behavior?

• 

  Xavier Noria August 1st, 2010 @ 07:41 PM

  Not sure.

  While it can't be said it is a bad practice per se, it is not
  something you want to encourage either. It could be the case that
  after Rails 3 lib is seen as a place where you put stuff that it is
  naturally required rather than autoloaded.

  The existence of autoload_paths is known I believe. It has been always
  there (under a different name). People may add lib to it as a quick
  migration fix, or they may take the opportunity to rethink the
  organization of their applications.

  To put it differently, I wouldn't use %W(#{config.root}/app/observers)
  either. Some people like that folder, some people don't care. It is
  not black and white and such example could send an implicit message in
  addition.

  Not against it, but not convinced either.

• You flagged this item as spam.
  

  Mateo Murphy August 1st, 2010 @ 08:08 PM

  It could be the case that after Rails 3 lib is seen as a place where you put stuff that it is naturally required rather than autoloaded.

  Sure, but then what do you do with stuff that you do want autoloaded, but doesn't fit in the app folders, like a form builder? From what I can see, I can:

  • Add lib to autoload
  • Stick it in one of the already autoloaded folders (but which one?)
  • Make a new folder and autoload that (app/lib??)
  • Explicitly require the form builder (but where? not in the view surely...)
  • Create a file (in initializers?) with some autoload statements

  But none of this seems to be what I'd consider a best practice, which is unfortunate because it's a need that comes up in all my rails projects.

• 

  Xavier Noria August 1st, 2010 @ 08:39 PM

  There's no magic bullet in my view. It depends on one's preferences
  and on every application's needs. That's why I said before that it is
  fine that people organize and tell Rails to autoload stuff as they
  prefer.

  I personally put form builders in app/helpers. Why? Well, they are
  conceptually related if you interpret app/helpers with some
  flexibility. AWDwR puts them there as well. I also put observers into
  app/models, because for me that's the place for all the M layer (which
  is broader than domain objects), and that organization is simple and
  enough for me unless the application is too big and needs more
  structure.

  That's debatable of course. Other people prefer other schemas. That's
  OK, if I do a code review and see app/observers no prob, it is also a
  valid choice. If they define app/form_builders, perfect.

  Ruby provides $: if you want to push stuff in there, and Rails
  additionally provides autoload_paths, so you can tailor this to your
  application and preferences.

  I only use lib for things that are transversal to the entire stack,
  utility methods over Strings... that kind of thing.

• You flagged this item as spam.
  

  Samuel Kadolph August 4th, 2010 @ 01:22 AM

  wycat has put lib back into the auto-loaded directories.

  I feel that files in lib should be auto-loaded because I've been hating having to move my namespaced (Mixins for example) pieces of code out of lib and into a new folder in app to end up with the namespace in the file structure twice (app/mixins/mixins/whatever.rb to get Mixins::Whatever to autoload.)

  Additionally, I don't think we should be eager-loading files in lib. It could end up being a significant slow down if you have enough files present that will have to be unloaded and loaded again each request (especially if it hits the DB).

• You flagged this item as spam.
  

  Akira Matsuda August 4th, 2010 @ 07:36 AM

  wycats' commit caused another problem.

  Since rake rails:templates:copy command copies the generators' templates (these files are named *.rb but actually not valid Ruby files but ERB files) into lib directory, now Rails fails when loading the copied templates in lib directory.

  Try the following steps to reproduce:

  1. create a new Edge Rails app

  2. copy templates
     

     % rake rails:templates:copy
     

  3. generate sth
     

     % rails g model user name:string
     

  4. BOOM!
     

     % rake
     

  /Users/a_matsuda/src/rails/activesupport/lib/active_support/dependencies.rb:219:in `require': /Users/a_matsuda/work/src/rails3/test/lib_autoload_problem/lib/templates/rails/controller/controller.rb:1: syntax error, unexpected '<' (SyntaxError)
  class <%= class_name %>Controller < ApplicationController
         ^
  /Users/a_matsuda/work/src/rails3/test/lib_autoload_problem/lib/templates/rails/controller/controller.rb:3: syntax error, unexpected tIDENTIFIER, expecting $end
    def <%= action %>
  

• You flagged this item as spam.
  

  Christian Seiler August 7th, 2010 @ 06:43 AM

  I think there really should be a standard folder in any Rails app (convention over configuration anyone?) which is autoloaded (and reloaded in dev mode) where I can put shared or abstracted code.

  Maybe app/lib (if lib causes problems)?

• You flagged this item as spam.
  

  RailsFanatic August 11th, 2010 @ 06:17 AM

  Yes Christian, I agree with you.

  Looks like Xavier is on the "me, me, me" track. "In my view" this and that.

  Yeah, let's find as many ways as possible to make migration to Rails 3 difficult for people. Give them more headaches than fewer, just based on Xavier's PERSONAL viewpoint that maybe a lib directory isn't such a good idea after all, in HIS view.

  Make these kinds of micro-decisions MORE difficult for Rails developers. Yeah. Just one more headache, when there are so many more important decisions to make. Good grief. Nothing better to do, eh?

  Really. Like Christian says, whatever happened to convention over configuration?

  Seems like this change adds more configuration and chastises anyone who favors the convention part.

  Don't ya love it when the Rails "gods" shove their views on everyone else just by making these kinds of changes without any community input???

• 

  Neeraj Singh August 11th, 2010 @ 06:35 AM

  @RailsFantaic

  To personally criticize others for not seeing your point of view is real cheap shot.

  Rails like any other open source project solicits input from community. However sometimes decisions are taken which a) may be not immediately evident why it was done b) may eventually turn out to be a wrong decision c) which is not what you expected. However a decision is to be taken to move forward.

  For you to personally single out "Xavier" who has contributed so much to Rails is highly unfortunate.

  Disagreements are bound to happen. That should not lead to loss of civility.

• 

  Neeraj Singh August 11th, 2010 @ 08:34 AM

  http://github.com/rails/rails/commit/02a5842cd09bd75de4c2fdb6b474c6...

• 

  Xavier Noria August 11th, 2010 @ 10:39 AM

  @RailsFanatic look, I have been only helping with the questions here. Given this patch is in, how to adapt. And my wording is relative because I am indeed explaining a point of view. It is not even a patch of mine!

• 

  Xavier Noria August 11th, 2010 @ 10:45 AM

  @Neeraj it has been reverted: http://github.com/rails/rails/commit/7e2399a42feb47407ad0cb06888158...

  There are technical difficulties, because you can't autoload in threadsafe mode (there are race conditions inherent to Ruby), and people put stuff there that prevents lib from being eager loaded. Like generators, rake tasks... Stuff that is not expected to run unless you run it.

  Let's see how this evolves.

• You flagged this item as spam.
  

  Christian Seiler August 11th, 2010 @ 11:16 AM

  @RailsFanatic: Thanks for seconding my statement, but I think we should keep arguing on a technical level.

  @xavier: Again, I really think there is a need for a standard folder, which is autoloaded and reloaded in dev mode. I guess almost every Rails app needs it. I don't quite get the arguments with thread-safety, what's the difference to app/*?

  We started a Rails 3 app based on RC 4 recently and updated to RC. Now our client asked us why the lib folder "doesn't work any longer". I told him to require the stuff. It didn't take long until the next question came along: "Why do I have to restart the server after every change?".

  Maybe it's just a configuration issue, but this one should be really solved by a convention without the need for making configuration changes to (virtually) every project.

  And it's not even about migrating a Rails 2 app. I'd be fine with a app/lib folder (for instance) instead.

• 

  Xavier Noria August 11th, 2010 @ 11:28 AM

  @Christian Yes I agree.

  The rationale for this change was not thread-safety. Please see the original patch.

  I am not defending this change, I understand the motivations, but the balance leans to keeping lib autoloaded for me.

  Now, again, given the patch is in, what to do? Christian in your case I'd leave everything as is and add lib to config.autoload_paths in config/application.rb.

  We are not yet in final, let's wait to see how it evolves.

• You flagged this item as spam.
  

  Christian Seiler August 11th, 2010 @ 01:04 PM

  Thanks for the workaround, it did the trick (though I strongly recommend to have such a folder by default, but I sound like a broken record).

• You flagged this item as spam.
  

  avramd September 16th, 2010 @ 07:26 PM

  • Assigned user changed from “José Valim” to “Xavier Noria”

  Hey Guys,

  I am having a problem with config.autoload_paths not working as described here. My libs are only getting loaded the first time they are used, and I have clarified this by putting Rails.logger calls in them and in my application_controller.rb - I see application_controller getting autoloaded each time, but my lib is not.

  Here is what I have added to config/application.rb, inside the definition of class Application.

  config.autoload_paths << File.join(config.root, "lib")
  

  Note that we have namespaced our app, so the actual definition looks as follows, don't know if that's a factor:

  module LWE
  class Application < Rails::Application

  # [... standard stuff omitted]
  config.autoload_paths << File.join(config.root, "lib")
  

• 

  Xavier Noria September 16th, 2010 @ 10:46 PM

  This is a minimal app that demonstrates that autoloading and reloading works

  fxn@halmos:~/tmp/autoload ∵ cat config/application.rb
  require File.expand_path('../boot', __FILE__)
  
  require 'rails/all'
  
  # If you have a Gemfile, require the gems listed there, including any gems
  # you've limited to :test, :development, or :production.
  Bundler.require(:default, Rails.env) if defined?(Bundler)
  
  module Autoload
    class Application < Rails::Application
      config.autoload_paths << "#{config.root}/lib"
    end
  end
  
  
  fxn@halmos:~/tmp/autoload ∵ cat config/routes.rb 
  Autoload::Application.routes.draw do
    root :to => lambda {|_| [200, {'Content-Type' => 'text/plain'}, Foo.ok]}
  end
  
  
  fxn@halmos:~/tmp/autoload ∵ cat lib/foo.rb 
  class Foo
    puts "loading Foo"
  
    def self.ok
      "OK\n"
    end
  end
  

  Could it be the case that some requests in your trials do not use the constant you are expecting to see autoloaded?

• You flagged this item as spam.
  

  eightbitraptor September 23rd, 2010 @ 08:28 AM

  Is there any call to have the rails app generator omit the lib directory because of this.

  If I am migrating across to Rails 3 and I see rails.root/lib I am going to assume that it's autoloaded. Making this change feel a little like a regression.

  I can submit a patch if necessary.

  thanks

  Matt

• 

  Xavier Noria September 23rd, 2010 @ 08:54 AM

  Not really, lib is a standard directory like public. Should be generated.

  A migration from Rails 2 to Rails 3 is a miniproject. There's a checklist. One of the tasks is to either add lib to autoload_paths in config/application.rb, or else revise the code to add require statements.

  The easiest solution is to add lib to autoload_paths. You're done. and the cost is that passenger won't eager load its contents for COW. Also, in a multithreaded application there's a little risk because autoloading is not thread-safe (if you hit the same thing concurrently, one thread may go on with a half-defined class). These may not be concerns for one's app though.

• 

  Xavier Noria September 23rd, 2010 @ 08:57 AM

  Ah, regarding COW, lib is neither eager loaded if it is not in autoload_paths. But since you eager load models etc. chances are that as a side-effect of their requires in practice you eager loaded most of lib.

• You flagged this item as spam.
  

  John G. October 8th, 2010 @ 03:29 PM

  Xavier,

  I'm noticing something similar as avramd, but it appears to only happen when patching Ruby classes. Here's an example:

  **config/application.rb'**

  module Autoload
    class Application < Rails::Application
      config.autoload_paths = %W(#{config.root/lib/ext)
    end
  end
  

  **lib/ext/time.rb:'**

  class Time
    def self.random(years_back=5)
      year = Time.now.year - rand(years_back) - 1
      month = rand(12) + 1
      day = rand(31) + 1
    end
  end
  

  When I fire up the rails console, typing Time.random gives me the following error:

  NoMethodError: undefined method 'random' for Time:Class
  

  To get this to work, I had to create an initializer and require the file

  config/initializers/load_ext.rb:

  require "lib/ext/time"
  

  After doing that, I am able to access Time.random from the Rails console.

  Is this how we were supposed to do it?

• 

  José Valim October 8th, 2010 @ 03:31 PM

  You cannot autoload things as Time, Object, etc. because autoload is only triggered if the constant is not defined yet. Since these are Ruby core, they are defined for sure.

• You flagged this item as spam.
  

  John G. October 8th, 2010 @ 06:06 PM

  Ah, that makes a lot of sense. Thanks for the clarification!

• You flagged this item as spam.
  

  avramd October 12th, 2010 @ 10:13 PM

  Xavier,

  With a little more experimenting, I have isolated the difference that makes our app fail/succeed to autoload. I still don't fully understand it because when I port this issue to your minimal app, it still works there.

  Nevertheless, the issue was that the module that was not getting autoreloaded had the same name as the application module - the one you get when you say "rails new foo". So with that example it seems we thought we were defining a module Foo, but because the application module Foo already existed, it seems we were merely extending it. Perhaps since it is already defined it does not get autoreloaded? I still don't understand why autoreloading did work when I modified your min app in this same way.

  I have a separate problem now with autoloading which may not be related to autoREloading - one of my lib files appears to not be getting auto-discovered. It is in the same directory as three others that are getting discovered fine. The problem file does not even get loaded the first time, unless I explicitly require it. It is in a subdirectory under lib.

  Would you mind pointing me to the section of code that actually iterates over lib and its contents so I can run through it in the debugger and try to better understand what's going on?

  Thanks,
  Avram

• 

  Xavier Noria October 12th, 2010 @ 10:17 PM

  @avramd interesting gotcha with the Foo reopening :).

  Actually lib is not scanned. Autoloading is lazy, if you try to use constant X, and X does not exist, there's a const_missing callback that gets called. That callback iterates over all the autoload_paths looking for a file called x.rb. If it finds it, it loads it.

  Are you using the constant that should trigger loading of that file you miss in lib?

• You flagged this item as spam.
  

  avramd October 12th, 2010 @ 10:18 PM

  I apologize.

  The problem was that the module was a "two-word" name, and the file name was not a correct de-camlization of it. I was trying to define module BarFoo in file bar-foo.rb instead of bar_foo.rb.

• You flagged this item as spam.
  

  John G. October 26th, 2010 @ 03:50 AM

  Xavier, Jose, could either of you comment on this issue I'm having: http://stackoverflow.com/questions/4018757/rails3-not-reloading-cod...

• You flagged this item as spam.
  

  John G. October 26th, 2010 @ 05:53 AM

  Nevermind. The issue has been resolved. =)

• 

  Jeff Kreeftmeijer November 7th, 2010 @ 04:52 PM

  • Tag cleared.

  Automatic cleanup of spam.

• You flagged this item as spam.
  

  jfelchner January 18th, 2011 @ 07:42 PM

  José, regarding what you said on 10/8, is there any chance we could get some sort of warning if a class is not loaded? I just spent about 45 minutes trying this before finding this post.

  Given that Ruby always allows you to reopen classes and define new functionality, you would expect that reopening any class in a path you've decided should be auto-loaded would work.

  I'm not saying it's wrong to do it as it is, I'm sure Rails core have their reasons, but just that something which goes against what a typical Ruby developer would be expecting might need to be specifically called out during load time.

  Thoughts?

• 

  Xavier Noria January 18th, 2011 @ 08:20 PM

  @jfelchner which code are you using?

  Note that the class keyword won't issue any const_missing. If you want to autoload and reopen at the same time then you need to use the constant somehow, for example with class_eval:

  MyModel.class_eval do
    # ...
  end
  

  There MyModel is an expression, so if the constant is unknown at that point dependencies will go look for it. If MyModel is found execution will follow, otherwise you'd get an (expected) NameError.